PLOTTING AMCAT DATA BY EXPLORATORY DATA ANALYSIS (EDA) USING MATPLOTLIB & SEABORN FROM PYTHON¶
GRAPHS: A diagram showing the relation between variable quantities, typically of two variables, each measured along one of a pair of axes at right angles.¶
Exploratory Data Analysis (EDA), is essentially a type of story telling for statisticians. It allows us to uncover patterns and insights, often with visual methods, within data.¶
STEP - I: IMPORTING THE NECESSARY LIBRARIES¶
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import plotly.express as px
NUMPY: It's a Python library used for working with arrays, also has functions for working in domain of linear algebra, fourier transform, and matrices. NumPy was created in 2005 by Travis Oliphant. It is an open source project and you can use it freely. NumPy stands for Numerical Python.¶
PANDAS: It's a software library written for the Python programming language for data manipulation and analysis. In particular, it offers data structures and operations for manipulating numerical tables and time series.¶
MATPLOTLIB: It's a plotting library for the Python programming language and its numerical mathematics extension NumPy. It provides an object-oriented API for embedding plots into applications using general-purpose GUI toolkits like Tkinter, wxPython, Qt, or GTK+..... SciPy makes use of Matplotlib.¶
SEABORN: It's a Python data visualization library based on matplotlib. It provides a high-level interface for drawing attractive and informative statistical graphics.¶
PLOTLY EXPRESS: Plotly Express is a new high-level Python visualization library, is a wrapper for Plotly.py that exposes a simple syntax for complex charts.¶
STEP - II: IMPORTING THE DATA¶
df = pd.read_excel(r'C:\Users\admin\Desktop\KANAV BANSAL\KANAV TASKS\TASK - II - EDA - AMCAT DATA/reports.xlsx')
df
Observation :¶
There are 3998 data of the people which contains the Gender, Designation, Salary, Location, Percentage of 10th, 12th and B.Tech including with College Tier -Specialization, etc things.
STEP - III: DATA ANALYSIS¶
include = 'all' is provided as an option, the result will include a union of attributes of each type. The includes and excludes parameters, can be used to limit which columns in a DataFrame are analyzed for the output. The parameters are ignored when analyzing a Series & describes a numeric Series.¶
df.describe(include='all')
Obesrvation:¶
We are able to get the information of MEAN, MEDIAN, STANDARD DEVIATION, MINIMUM & MAXIMUM VALUES OF ENTIRE DATA.
GETTING THE TOP 5 & LAST 5 DATA¶
X = df.head()
X
OBSERVATION:¶
There are 3 males & 2 females, where one of the male has highest Salary Package who is a SENIOR SOFTWARE ENGINEER.
Y = df.tail()
Y
Observation:¶
There are 3 females & 2 males, where one of the female has highest Salary Package who is a SENIOR SYSTEMS ENGINEER.
STEP - IV: PLOTTING THE DATA¶
DISTPLOT: Seaborn distplot lets you show a histogram with a line on it. We use seaborn in combination with matplotlib, the Python plotting module. It plots a UNIVARIATE distribution of observations. The distplot() function combines the matplotlib hist function with the seaborn kdeplot() and rugplot() functions.¶
KDE PLOT: kernel density estimation (KDE) is a non-parametric way to estimate the probability density function of a random variable. Kernel density estimation is a fundamental data smoothing problem where inferences about the population are made, based on a finite data sample and Show a univariate or bivariate distribution.¶
RUG PLOT: A rug plot is a plot of data for a single quantitative variable, displayed as marks along an axis. It is used to visualise the distribution of the data. As such it is analogous to a histogram with zero-width bins, or a one-dimensional scatter plot. Draws a small vertical lines to show each observation in a distribution.¶
sns.distplot(df['Salary'])
sns.distplot(df['Salary'], kde = False, rug = True)
sns.distplot(df['Salary'], kde = False, rug = False)
sns.distplot(df['collegeGPA'], kde = False, rug = True)
sns.distplot(df['Logical'], kde = False, rug = True)
Observation:¶
It's a figure-level function with a similar flexibility over the kind of plot to draw.
It's basically for univariant set of observations and visualizes it through a histogram i.e. only one observation and hence we choose one particular column of the dataset.
The plot shows a simple distribution when it creats a random values with random.randn().
POINT PLOT: A point plot represents an estimate of central tendency for a numeric variable by the position of scatter plot points and provides some indication of the uncertainty around that estimate using error bars.¶
CATPLOT: It's a relatively new addition to Seaborn that simplifies plotting that involves categorical variables. Catplot function can do all these types of plots and one can specify the type of plot one needs with the kind parameter. The default kind in catplot() is “strip”, corresponding to stripplot(). Combines a categorical plot with a FacetGrid.¶
sns.catplot(x = "Gender", y = "Salary", hue = "CollegeTier", kind = "point", data = df)
sns.catplot(x = "Gender", y = "Salary", hue = "Degree", kind = "point", data = df)
sns.catplot(x="collegeGPA", y="Salary", hue="Gender", markers=["^", "o"], linestyles=["-", "--"], kind="point", data = X)
sns.catplot(x="collegeGPA", y="Salary", hue="Gender", markers=["^", "o"], linestyles=["-", "--"], kind="point", data = Y)
sns.catplot(x="collegeGPA", y="Salary", hue="Gender", markers=["^", "o"], linestyles=["-", "--"], kind="point", data=df)
sns.catplot(x="Degree", y="Salary", hue="Gender", markers=["^", "o"], linestyles=["-", "--"], kind="point", data = df)
Observation:¶
Point plots can be more useful than bar plots for focusing comparisons between different levels of one or more categorical variables.
They are particularly adept at showing interactions: how the relationship between levels of one categorical variable changes across levels of a second categorical variable. The lines that join each point from the same hue level allow interactions to be judged by differences in slope, which is easier for the eyes than comparing the heights of several groups of points or bars.
It is important to keep in mind that a point plot shows only the mean (or other estimator) value, but in many cases it may be more informative to show the distribution of values at each level of the categorical variables. In that case, other approaches such as a box or violin plot may be more appropriate.
VIOLIN PLOT: A violin plot gives a combination of boxplot and kernel density estimate, plays a similar role as a box and whisker plot. It shows the distribution of quantitative data across several levels of one (or more) categorical variables such that those distributions can be compared.¶
sns.violinplot(x = df.CollegeTier, y = df.collegeGPA)
sns.violinplot(x = df.Degree, y = df.collegeGPA)
sns.catplot(x= "Degree", y= "collegeGPA", hue= "Gender", kind= "violin", inner= "stick", split=True, palette="pastel", data= df)
sns.catplot(x = "Degree", y = "collegeGPA", hue = "Gender", kind = "violin", split = True, data = df)
sns.catplot(x = "collegeGPA", y = "Degree", hue = "Gender", kind = "violin", bw = .15, cut = 0, data = df)
sns.catplot(x = "Degree", y = "collegeGPA", hue = "Gender", kind = "violin", data = df)
Observation:¶
The white dot represents the median, the thick gray bar in the center represents the interquartile range, the thin gray line represents the rest of the distribution, except for points that are determined to be “outliers” using a method that is a function of the interquartile range.
Violin plot are made vertically most of the time, If you have long labels building an horizontal version{Output[22]} like above make the labels more readable.
If the variable are grouped, we can build a grouped violin as you would do for a boxplot.
BOXPLOT: A Boxplot is a very basic plot - used to visualize distributions. It's very useful when you want to compare data between two groups. Sometimes a boxplot is named a box-and-whisker plot. Any box shows the quartiles of the dataset while the whiskers extend to show the rest of the distribution.¶
sns.catplot(x = "Degree", y = "Salary", kind = "boxen", data = df.sort_values("collegeGPA"))
sns.catplot(x = "Degree", y = "Salary", hue = "Gender", kind = "box", data = df)
sns.boxplot(data = X, x='Specialization', y='Salary')
sns.boxplot(data = df, x = 'Specialization', y = 'Salary')
sns.boxplot(data = df, x = 'Designation', y = 'Salary')
sns.boxplot(data = df, x = 'GraduationYear', y = 'Salary')
sns.boxplot(data = df, x = 'CollegeTier', y = 'Salary')
sns.boxplot(data = df, x = 'CollegeTier', y = 'Degree')
sns.boxplot(data = X, x = 'collegeGPA', y = 'Designation')
sns.boxplot(data = df, x = 'Degree', y = 'Salary')
sns.boxplot(data = X, x = 'Salary', y = 'Designation', hue = 'Specialization')
sns.boxplot(data = Y, x = 'Specialization', y = 'Salary', hue = 'Designation')
sns.catplot(x = "English", y = "Logical", kind = "boxen", data = df.sort_values("Gender"))
plt.boxplot(df['Salary'])
plt.show()
plt.boxplot(df['collegeGPA'])
plt.show()
Observation:¶
OUTLIERS: An outlier is an observation that is numerically distant from the rest of the data. Box plots are useful as they show outliers within a data set. When reviewing a box plot, an outlier is defined as a data point that is located outside the whiskers of the box plot.
Comparision of the medians, the interquartile ranges and whiskers of box plots.
Gives the potential outliers and signs of Skewness.
SWARM PLOT: This style of plot is sometimes called a “beeswarm”. A swarm plot can be drawn on its own, but it is also a good complement to a box or violin plot in cases where you want to show all observations along with some representation of the underlying distribution.¶
sns.catplot(x = "Gender", y = "Salary", order = ["m", "f"], data = df)
sns.catplot(x = "English", y = "Quant", hue = "Gender", kind = "swarm", data = df)
sns.catplot(x = "GraduationYear", y = "collegeGPA", data = df)
sns.catplot(x = "Degree", y = "Salary", hue = "Gender", kind = "swarm", data = df)
sns.catplot(x = "CollegeTier", y = "collegeGPA", kind = "swarm", data = df)
sns.swarmplot(data = df, x = 'Specialization', y = 'Salary')
sns.swarmplot(data = df, x = 'Domain', y = 'English')
sns.swarmplot(data = df, x = 'Salary', y = 'Domain')
Observation:¶
It can give a better representation of the distribution of observations, although it only works well for relatively small datasets.
Enlarging the plot and Separate points by hue using the argument split = True.
Place the legend to the right while Adjusting the y-axis limits to end at 0.
SCATTER PLOT: They can plot two-dimensional graphics that can be enhanced by mapping up to three additional variables while using the semantics of hue, size, and style parameters. Using redundant semantics can be helpful for making graphics more accessible.¶
JOINT PLOT: It displays a relationship between 2 variables (bivariate) as well as 1D profiles (univariate) in the margins. This plot is a convenience class that wraps JointGrid.¶
sns.jointplot(x = 'Salary', y = 'collegeGPA', data = df, kind = 'scatter')
sns.jointplot(x = 'collegeGPA', y = 'Salary', data = df, kind = 'scatter')
sns.jointplot(x = 'English', y = 'Logical', data = df, kind = 'scatter')
sns.jointplot(x = 'English', y = 'collegeGPA', data = df, kind = 'scatter')
sns.jointplot(x='Logical', y='Quant', data=df, kind = 'scatter')
plt.scatter(df['Salary'], df['collegeGPA'])
plt.show()
plt.scatter(df['Salary'], df['Specialization'])
plt.show()
plt.scatter(df['Gender'], df['Salary'])
plt.show()
plt.scatter(df['English'], df['Domain'])
plt.show()
plt.scatter(df['Quant'], df['Logical'])
plt.show()
plt.scatter(df['Degree'], df['CollegeState'])
plt.show()
plt.scatter(df['Degree'], df['CollegeTier'])
plt.show()
plt.scatter(df['collegeGPA'], df['Gender'])
plt.show()
Observation:¶
SCATTER PLOT:¶
Pairs of numerical figures are present.
Dependent variables have multiple values for each figure associated with the independent variable.
Defining if there is a relationship between two variables and only show correlation.
Discrete data is best at pass/ fail measurements, Continuous data lets you measure things deeply on an infinite set and is generally used in scatter analysis.
JOINT PLOT:¶
From the output, you can see that a joint plot has three parts.
A distribution plot at the top for the column on the x-axis, a distribution plot on the right for the column on the y-axis and a scatter plot in between that shows the mutual distribution of data for both the columns.
You can see that there is no correlation observed between the x, y variables as given in the input.
You can change the type of the joint plot by passing a value for the kind parameter.
HEXBIN PLOT: A Hexbin plot is useful to represent the relationship of 2 numerical variables when you have a lot of data point. Instead of overlapping, the plotting window is split in several hexbins, and the number of points per hexbin is counted. The color denotes this number of points.¶
sns.jointplot(x = 'collegeGPA', y = 'Salary', data = df, kind = 'hex', color = 'k')
sns.jointplot(x = 'English', y = 'Quant', data = df, kind = 'hex', color = 'b')
sns.jointplot(x = 'CollegeTier', y = 'Domain', data = X, kind = 'hex', color = 'b')
sns.jointplot(x = 'Quant', y = 'Logical', data = df, kind = 'hex', color = 'b')
sns.jointplot(x = 'CollegeID', y = 'Salary', data = df, kind = 'hex', color = 'b')
Observation:¶
Instead of overlapping, the plotting window is split in several hexbins, and the number of points per hexbin is counted.
The color denotes this number of points.
The size of the hexagons changes - the scale of the color bar guide is redefined accordingly and We can change the size of the bins using the gridsize argument.
We get a clear picture of density, distributions, and relative ranges, similar to a heat map.
The shape of the hexagon allows us to limit the effects of edge biases found in square bins, while retaining the ability to form a continuous grid.
PAIR PLOT: Pair Plots are a really simple (one-line-of-code simple!) way to visualize relationships between each variable. It produces a matrix of relationships between each variable in your data for an instant examination of our data. It can also be a great jumping off point for determining types of regression analysis to use.¶
sns.pairplot(df)
